Inferring Decision Trees Using the Minimum Description Length Principle

This paper concerns methods for inferring decision trees from examples for classification problems. The reader who is unfamiliar with this problem may wish to consult J. R. Quinlan’s paper (1986), or the excellent monograph by Breiman et al. (1984), although this paper will be self-contained. This work is inspired by Rissanen’s work on the Minimum description length principle (or MDLP for short) and on his related notion of the stochastic complexity of a string Rissanen, 1986b. The reader may also want to refer to related work by Boulton and Wallace (1968, 1973a, 1973b), Georgeff and Wallace (1984), and Hart (1987). Roughly speaking, the minimum description length principle states that the best “theory” to infer from a set of data is the one which minimizes the sum of